{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ace41c03-dfa3-4cb6-88bc-bcaa72cfdc85",
   "metadata": {},
   "source": [
    "# RNN实现情感分类\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1823275c-96a6-4c12-839c-623ac2662c6c",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 概述\n",
    "\n",
    "情感分类是自然语言处理中的经典任务，是典型的分类问题。本案例使用MindSpore实现一个基于RNN网络的情感分类模型，实现如下的效果：\n",
    "\n",
    "```text\n",
    "输入: This film is terrible\n",
    "正确标签: Negative\n",
    "预测标签: Negative\n",
    "\n",
    "输入: This film is great\n",
    "正确标签: Positive\n",
    "预测标签: Positive\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0df2583",
   "metadata": {},
   "source": [
    "## 环境准备\n",
    "\n",
    "开发者拿到香橙派开发板后，首先需要进行硬件资源确认，镜像烧录及CANN和MindSpore版本的升级，才可运行该案例，具体如下：\n",
    "\n",
    "- 硬件： 香橙派AIpro 8G 8T开发板\n",
    "- 镜像： 香橙派官网ubuntu镜像\n",
    "- CANN：8.1.RC1\n",
    "- MindSpore： 2.6.0\n",
    "\n",
    "### 镜像烧录\n",
    "\n",
    "运行该案例需要烧录香橙派官网ubuntu镜像，烧录流程参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--镜像烧录](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#1-%E9%95%9C%E5%83%8F%E7%83%A7%E5%BD%95%E4%BB%A5windows%E7%B3%BB%E7%BB%9F%E4%B8%BA%E4%BE%8B)章节。\n",
    "\n",
    "### CANN升级\n",
    "\n",
    "CANN升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--CANN升级](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#3-cann%E5%8D%87%E7%BA%A7)章节。\n",
    "\n",
    "### MindSpore升级\n",
    "\n",
    "MindSpore升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--MindSpore升级](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#4-mindspore%E5%8D%87%E7%BA%A7)章节。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72464b8e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/miniconda3/envs/ms/lib/python3.10/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.\n",
      "  setattr(self, word, getattr(machar, word).flat[0])\n",
      "/usr/local/miniconda3/envs/ms/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.\n",
      "  return self._float_to_str(self.smallest_subnormal)\n",
      "/usr/local/miniconda3/envs/ms/lib/python3.10/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.\n",
      "  setattr(self, word, getattr(machar, word).flat[0])\n",
      "/usr/local/miniconda3/envs/ms/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.\n",
      "  return self._float_to_str(self.smallest_subnormal)\n"
     ]
    }
   ],
   "source": [
    "import os, shutil, tempfile, zipfile, requests\n",
    "from typing import IO, Dict, Any, List\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "\n",
    "import mindspore as ms\n",
    "import mindspore.dataset as ds\n",
    "from mindspore import mint\n",
    "import mindspore.nn as nn\n",
    "ms.device_context.ascend.op_precision.precision_mode(\"force_fp16\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bd4ac0d-886b-44a0-b794-ef7c4d867a3e",
   "metadata": {},
   "source": [
    "## 数据处理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeded00a-794c-49f0-9d06-c1c43374bc78",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 数据下载模块\n",
    "\n",
    "为了方便预训练词向量的下载，首先设计数据下载模块，实现可视化下载流程，并保存至指定路径。数据下载模块使用`requests`库进行http请求，并通过`tqdm`库对下载百分比进行可视化。此外针对下载安全性，使用IO的方式下载临时文件，而后保存至指定的路径并返回。\n",
    "\n",
    "> `tqdm`和`requests`库需手动安装，命令如下：`pip install tqdm requests`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0f145ba5-5259-4520-855f-691a76e93bac",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/\n",
      "Requirement already satisfied: tqdm in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (4.67.1)\n",
      "Requirement already satisfied: requests in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (2.32.4)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests) (3.4.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests) (3.10)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests) (2.5.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests) (2025.8.3)\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.\u001b[0m\u001b[33m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "#install tqdm and requests\n",
    "\n",
    "!pip install tqdm requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a11d2a34-fdea-4731-ac96-31968c23d575",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "cache_dir = os.path.join(os.getcwd(), \"mindspore_examples\")\n",
    "os.makedirs(cache_dir, exist_ok=True)\n",
    "\n",
    "def _http_get(url: str, temp_file: IO):\n",
    "    r = requests.get(url, stream=True, timeout=60)\n",
    "    total = int(r.headers.get(\"Content-Length\") or 0)\n",
    "    bar = tqdm(total=total, unit=\"B\", desc=\"file_sizes\")\n",
    "    for ch in r.iter_content(1024 * 256):\n",
    "        if ch:\n",
    "            temp_file.write(ch); bar.update(len(ch))\n",
    "    bar.close()\n",
    "\n",
    "def downloads(file_name: str, url: str):\n",
    "    dst = os.path.join(cache_dir, file_name)\n",
    "    if not os.path.exists(dst):\n",
    "        with tempfile.NamedTemporaryFile() as t:\n",
    "            _http_get(url, t); t.flush(); t.seek(0)\n",
    "            with open(dst, \"wb\") as f: shutil.copyfileobj(t, f)\n",
    "    return dst"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45ee5b20-7547-4ca1-83fa-6ee083cc65a4",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 加载预训练词向量\n",
    "\n",
    "预训练词向量是对输入单词的数值化表示，通过`nn.Embedding`层，采用查表的方式，输入单词对应词表中的index，获得对应的表达向量。\n",
    "因此进行模型构造前，需要将Embedding层所需的词向量和词表进行构造。这里我们使用Glove(Global Vectors for Word Representation)这种经典的预训练词向量，\n",
    "其数据格式如下：\n",
    "\n",
    "| Word |  Vector |  \n",
    "|:---|:---:|\n",
    "| the | 0.418 0.24968 -0.41242 0.1217 0.34527 -0.044457 -0.49688 -0.17862 -0.00066023 ...|\n",
    "| , | 0.013441 0.23682 -0.16899 0.40951 0.63812 0.47709 -0.42852 -0.55641 -0.364 ... |\n",
    "\n",
    "我们直接使用第一列的单词作为词表，使用`dataset.text.Vocab`将其按顺序加载；同时读取每一行的Vector并转为`numpy.array`，用于`nn.Embedding`加载权重使用。具体实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "519a863f-4053-4c20-93e2-b0acac829372",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def load_glove(glove_zip_path: str):\n",
    "    glove_100d = os.path.join(cache_dir, \"glove.6B.100d.txt\")\n",
    "    if not os.path.exists(glove_100d):\n",
    "        with zipfile.ZipFile(glove_zip_path) as zf: zf.extractall(cache_dir)\n",
    "\n",
    "    embs, toks = [], []\n",
    "    with open(glove_100d, encoding=\"utf-8\") as gf:\n",
    "        for line in gf:\n",
    "            w, vec = line.split(maxsplit=1)\n",
    "            toks.append(w)\n",
    "            embs.append(np.fromstring(vec, dtype=np.float32, sep=\" \"))\n",
    "    embs.append(np.random.rand(100).astype(np.float32))  # <unk>\n",
    "    embs.append(np.zeros((100,), np.float32))            # <pad>\n",
    "\n",
    "    vocab = ds.text.Vocab.from_list(toks, special_tokens=[\"<unk>\", \"<pad>\"], special_first=False)\n",
    "    embs = np.asarray(embs, np.float32).astype(np.float16)  # 直接转 FP16\n",
    "    return vocab, embs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fa216e1-db92-4c74-868a-b0a1889810f3",
   "metadata": {},
   "source": [
    "由于数据集中可能存在词表没有覆盖的单词，因此需要加入`<unk>`标记符；同时由于输入长度的不一致，在打包为一个batch时需要将短的文本进行填充，因此需要加入`<pad>`标记符。完成后的词表长度为原词表长度+2。\n",
    "\n",
    "下面下载Glove词向量，并加载生成词表和词向量权重矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "34ca0aee-7deb-45ea-8231-2be5f9c2e632",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "400002"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "glove_path = downloads(\"glove.6B.zip\",\n",
    "    \"https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/glove.6B.zip\")\n",
    "vocab, embeddings = load_glove(glove_path)\n",
    "len(vocab.vocab())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f199e78-e711-4801-87f0-a099289bdfc1",
   "metadata": {},
   "source": [
    "使用词表将`the`转换为index id，并查询词向量矩阵对应的词向量："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b12e8554-496c-4f35-a0bc-67a3f60ff488",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0,\n",
       " array([-0.0382 , -0.2449 ,  0.728  , -0.3997 ,  0.0832 ,  0.04395,\n",
       "        -0.3914 ,  0.3345 , -0.5757 ,  0.08746,  0.2878 , -0.0673 ,\n",
       "         0.309  , -0.264  , -0.1323 , -0.2075 ,  0.334  , -0.3384 ,\n",
       "        -0.3174 , -0.4834 ,  0.1464 , -0.373  ,  0.3457 ,  0.05203,\n",
       "         0.4495 , -0.4697 ,  0.02628, -0.5415 , -0.1552 , -0.1411 ,\n",
       "        -0.03973,  0.2827 ,  0.1439 ,  0.2346 , -0.3103 ,  0.0862 ,\n",
       "         0.204  ,  0.5264 ,  0.1716 , -0.0824 , -0.718  , -0.4153 ,\n",
       "         0.2034 , -0.1277 ,  0.4136 ,  0.552  ,  0.579  , -0.3347 ,\n",
       "        -0.3655 , -0.5483 , -0.06287,  0.2659 ,  0.302  ,  0.9976 ,\n",
       "        -0.8047 , -3.023  ,  0.01254, -0.3694 ,  2.217  ,  0.722  ,\n",
       "        -0.2498 ,  0.9214 ,  0.03452,  0.4675 ,  1.107  , -0.1936 ,\n",
       "        -0.0746 ,  0.2335 , -0.05206, -0.2205 ,  0.05716, -0.1581 ,\n",
       "        -0.3079 , -0.4163 ,  0.3796 ,  0.15   , -0.532  , -0.2054 ,\n",
       "        -1.253  ,  0.0716 ,  0.7056 ,  0.4976 , -0.4207 ,  0.2615 ,\n",
       "        -1.538  , -0.3022 , -0.0734 , -0.2832 ,  0.371  , -0.2522 ,\n",
       "         0.01622, -0.0171 , -0.39   ,  0.874  , -0.7256 , -0.5107 ,\n",
       "        -0.5205 , -0.1459 ,  0.8276 ,  0.2705 ], dtype=float16))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idx = vocab.tokens_to_ids('the')\n",
    "embedding = embeddings[idx]\n",
    "idx, embedding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfab4316-c35d-430d-bfaa-7af9b4f5b736",
   "metadata": {},
   "source": [
    "## 模型构建\n",
    "\n",
    "用于情感分类的模型结构，首先需要将输入文本(即序列化后的index id列表)通过查表转为向量化表示，此时需要使用`nn.Embedding`层加载Glove词向量；然后使用RNN循环神经网络做特征提取；最后将RNN连接至一个全连接层，即`nn.Dense`，将特征转化为与分类数量相同的size，用于后续进行模型优化训练。整体模型结构如下：\n",
    "\n",
    "```text\n",
    "nn.Embedding -> nn.RNN -> nn.Dense\n",
    "```\n",
    "\n",
    "这里我们使用能够一定程度规避RNN梯度消失问题的变种LSTM(Long short-term memory)做特征提取层。下面对模型进行详解："
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f64cdfd1-4fef-4394-a4ea-e5e7b509a9cc",
   "metadata": {},
   "source": [
    "### Embedding\n",
    "\n",
    "Embedding层又可称为EmbeddingLookup层，其作用是使用index id对权重矩阵对应id的向量进行查找，当输入为一个由index id组成的序列时，则查找并返回一个相同长度的矩阵，例如：\n",
    "\n",
    "```text\n",
    "embedding = nn.Embedding(1000, 100) # 词表大小(index的取值范围)为1000，表示向量的size为100\n",
    "input shape: (1, 16)                # 序列长度为16\n",
    "output shape: (1, 16, 100)\n",
    "```\n",
    "\n",
    "这里我们使用前文处理好的Glove词向量矩阵，设置`nn.Embedding`的`embedding_table`为预训练词向量矩阵。对应的`vocab_size`为词表大小400002，`embedding_size`为选用的`glove.6B.100d`向量大小，即100。\n",
    "\n",
    "### RNN(循环神经网络)\n",
    "\n",
    "循环神经网络（Recurrent Neural Network, RNN）是一类以序列（sequence）数据为输入，在序列的演进方向进行递归（recursion）且所有节点（循环单元）按链式连接的神经网络。下图为RNN的一般结构：\n",
    "\n",
    "![RNN-0](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.2/tutorials/application/source_zh_cn/nlp/images/0-RNN-0.png)\n",
    "\n",
    "> 图示左侧为一个RNN Cell循环，右侧为RNN的链式连接平铺。实际上不管是单个RNN Cell还是一个RNN网络，都只有一个Cell的参数，在不断进行循环计算中更新。\n",
    "\n",
    "由于RNN的循环特性，和自然语言文本的序列特性(句子是由单词组成的序列)十分匹配，因此被大量应用于自然语言处理研究中。下图为RNN的结构拆解：\n",
    "\n",
    "![RNN](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.2/tutorials/application/source_zh_cn/nlp/images/0-RNN.png)\n",
    "\n",
    "RNN单个Cell的结构简单，因此也造成了梯度消失(Gradient Vanishing)问题，具体表现为RNN网络在序列较长时，在序列尾部已经基本丢失了序列首部的信息。为了克服这一问题，LSTM(Long short-term memory)被提出，通过门控机制(Gating Mechanism)来控制信息流在每个循环步中的留存和丢弃。下图为LSTM的结构拆解：\n",
    "\n",
    "![LSTM](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r2.2/tutorials/application/source_zh_cn/nlp/images/0-LSTM.png)\n",
    "\n",
    "本案例我们选择LSTM变种而不是经典的RNN做特征提取，来规避梯度消失问题，并获得更好的模型效果。下面来看MindSpore中`nn.LSTM`对应的公式：\n",
    "\n",
    "$$h_{0:t}, (h_t, c_t) = \\text{LSTM}(x_{0:t}, (h_0, c_0))$$\n",
    "\n",
    "这里`nn.LSTM`隐藏了整个循环神经网络在序列时间步(Time step)上的循环，送入输入序列、初始状态，即可获得每个时间步的隐状态(hidden state)拼接而成的矩阵，以及最后一个时间步对应的隐状态。我们使用最后的一个时间步的隐状态作为输入句子的编码特征，送入下一层。\n",
    "\n",
    "> Time step：在循环神经网络计算的每一次循环，成为一个Time step。在送入文本序列时，一个Time step对应一个单词。因此在本例中，LSTM的输出$h_{0:t}$对应每个单词的隐状态集合，$h_t$和$c_t$对应最后一个单词对应的隐状态。\n",
    "\n",
    "### Dense\n",
    "\n",
    "在经过LSTM编码获取句子特征后，将其送入一个全连接层，即`nn.Dense`，将特征维度变换为二分类所需的维度1，经过Dense层后的输出即为模型预测结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0f1e26d-1b7c-4d50-947d-4fdda527b651",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class RNN(nn.Cell):\n",
    "    def __init__(self, emb_table: np.ndarray, hidden: int, num_layers: int, pad_idx: int):\n",
    "        super().__init__()\n",
    "        vocab_size, emb_dim = emb_table.shape\n",
    "        self.embedding = mint.nn.Embedding(\n",
    "            num_embeddings=vocab_size,\n",
    "            embedding_dim=emb_dim,\n",
    "            padding_idx=pad_idx,\n",
    "            _weight=ms.Tensor(emb_table, ms.float16),\n",
    "            _freeze=False, dtype=ms.float16\n",
    "        )\n",
    "\n",
    "        self.hidden = hidden\n",
    "        self.num_layers = num_layers\n",
    "        in_sizes = [emb_dim] + [2*hidden] * (num_layers - 1)\n",
    "\n",
    "        self.fwd_cells = nn.CellList([\n",
    "            nn.LSTMCell(input_size=in_sizes[i], hidden_size=hidden, dtype=ms.float16)\n",
    "            for i in range(num_layers)\n",
    "        ])\n",
    "        self.bwd_cells = nn.CellList([\n",
    "            nn.LSTMCell(input_size=in_sizes[i], hidden_size=hidden, dtype=ms.float16)\n",
    "            for i in range(num_layers)\n",
    "        ])\n",
    "\n",
    "        self.fc = mint.nn.Linear(in_features=2*hidden, out_features=1, bias=True, dtype=ms.float16)\n",
    "\n",
    "    def _run_one_layer_dir(self, x_seq: ms.Tensor, cell: nn.Cell, reverse: bool):\n",
    "        B, T, _ = x_seq.shape\n",
    "        outs = [None] * T\n",
    "        h = ms.ops.zeros((B, self.hidden), ms.float16)\n",
    "        c = ms.ops.zeros((B, self.hidden), ms.float16)\n",
    "        idx = range(T-1, -1, -1) if reverse else range(T)\n",
    "        for t in idx:\n",
    "            xt = x_seq[:, t, :]\n",
    "            h, c = cell(xt, (h, c))\n",
    "            outs[t] = h \n",
    "        y_seq = mint.stack(outs, dim=1)   # (B,T,H)\n",
    "        h_last = outs[0] if reverse else outs[T-1]\n",
    "        return y_seq, h_last\n",
    "\n",
    "    def construct(self, ids: ms.Tensor):\n",
    "        # ids: (B,T) int32\n",
    "        seq_in = self.embedding(ids).astype(ms.float16)  # (B,T,E)\n",
    "        h_f_last = h_b_last = None\n",
    "\n",
    "        for l in range(self.num_layers):\n",
    "            y_f, h_f = self._run_one_layer_dir(seq_in, self.fwd_cells[l], reverse=False)  # (B,T,H)\n",
    "            y_b, h_b = self._run_one_layer_dir(seq_in, self.bwd_cells[l], reverse=True)   # (B,T,H)\n",
    "            if l < self.num_layers - 1:\n",
    "                seq_in = mint.cat((y_f, y_b), dim=2)\n",
    "            else:\n",
    "                h_f_last, h_b_last = h_f, h_b\n",
    "\n",
    "        feat = mint.cat((h_f_last, h_b_last), dim=1)  # (B,2H)\n",
    "        logits = self.fc(feat)                        # (B,1)  FP16\n",
    "        return logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "591b1301-e9e3-44eb-a88b-09d2e807f9f9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(202451:255086384599072,MainProcess):2025-08-09-04:28:46.653.000 [mindspore/nn/layer/rnn_cells.py:68] LSTMCell has been changed from 'single LSTM layer' to 'single LSTM cell', if you still need use single LSTM layer, please use `nn.LSTM` instead.\n",
      "[WARNING] ME(202451:255086384599072,MainProcess):2025-08-09-04:28:46.716.000 [mindspore/nn/layer/rnn_cells.py:68] LSTMCell has been changed from 'single LSTM layer' to 'single LSTM cell', if you still need use single LSTM layer, please use `nn.LSTM` instead.\n",
      "[WARNING] ME(202451:255086384599072,MainProcess):2025-08-09-04:28:46.841.000 [mindspore/nn/layer/rnn_cells.py:68] LSTMCell has been changed from 'single LSTM layer' to 'single LSTM cell', if you still need use single LSTM layer, please use `nn.LSTM` instead.\n",
      "[WARNING] ME(202451:255086384599072,MainProcess):2025-08-09-04:28:46.903.000 [mindspore/nn/layer/rnn_cells.py:68] LSTMCell has been changed from 'single LSTM layer' to 'single LSTM cell', if you still need use single LSTM layer, please use `nn.LSTM` instead.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RNN(\n",
       "  (embedding): EmbeddingExt(num_embeddings=400002, embedding_dim=100, padding_idx=400001, max_norm=None, norm_type=2.0, scale_grad_by_freq=False, dtype=Float16)\n",
       "  (fwd_cells): CellList(\n",
       "    (0): LSTMCell()\n",
       "    (1): LSTMCell()\n",
       "  )\n",
       "  (bwd_cells): CellList(\n",
       "    (0): LSTMCell()\n",
       "    (1): LSTMCell()\n",
       "  )\n",
       "  (fc): Linear(input_features=512, output_features=1, has_bias=True)\n",
       ")"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hidden_size = 256\n",
    "num_layers  = 2\n",
    "pad_idx     = vocab.tokens_to_ids(\"<pad>\")\n",
    "model = RNN(embeddings, hidden=hidden_size, num_layers=num_layers, pad_idx=pad_idx)\n",
    "model.set_train(False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ccc7b0d-9055-4cab-9b6d-46bc0b06f60a",
   "metadata": {},
   "source": [
    "## 模型加载\n",
    "使用MindSpore提供的Checkpoint加载和网络权重加载接口：1.将保存的模型Checkpoint加载到内存中，2.将Checkpoint加载至模型。\n",
    "\n",
    "> `load_param_into_net`接口会返回模型中没有和Checkpoint匹配的权重名，正确匹配时返回空列表。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e1642eaf-0d7e-478b-88cf-8095a18223c5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/\n",
      "Requirement already satisfied: download in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (0.3.5)\n",
      "Requirement already satisfied: tqdm in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from download) (4.67.1)\n",
      "Requirement already satisfied: six in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from download) (1.17.0)\n",
      "Requirement already satisfied: requests in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from download) (2.32.4)\n",
      "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests->download) (3.4.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests->download) (3.10)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests->download) (2.5.0)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/miniconda3/envs/ms/lib/python3.10/site-packages (from requests->download) (2025.8.3)\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.\u001b[0m\u001b[33m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "#install download\n",
    "\n",
    "!pip install download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d477ec12-04ff-488b-8d31-ae82e5e9be2f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://cdn.modelers.cn/lfs/b1/d6/c2ab538684d08d744bd6b262bdc243e00ad2c0af819e429c9e7574370e3a?response-content-disposition=attachment%3B+filename%3D%22sentiment-analysis.ckpt%22&AWSAccessKeyId=HAZQA0Q6AQL2GHX4TKTL&Expires=1754771330&Signature=MCyh%2BgVbrkKSs8BTnLnBhz%2F4fVo%3D (161.4 MB)\n",
      "\n",
      "file_sizes: 100%|████████████████████████████| 169M/169M [00:02<00:00, 76.6MB/s]\n",
      "Successfully downloaded file to ./sentiment-analysis.ckpt\n",
      "OK: 已把 nn.LSTM 的 ckpt 映射加载到 LSTMCell BiLSTM（直接 set_data，避免名字前缀问题）。\n"
     ]
    }
   ],
   "source": [
    "# download ckpt\n",
    "from download import download\n",
    "rnn_url = \"https://modelers.cn/coderepo/web/v1/file/MindSpore-Lab/cluoud_obs/main/media/examples/mindspore-courses/orange-pi-online-infer/07-RNN/sentiment-analysis.ckpt\"\n",
    "path = \"./sentiment-analysis.ckpt\"\n",
    "ckpt_path = download(rnn_url, path, replace=True)\n",
    "def _to_np(x): return x.asnumpy() if hasattr(x, \"asnumpy\") else np.asarray(x)\n",
    "\n",
    "def load_lstm_ckpt_into_cell_bilstm(model: RNN, ckpt_path: str, strict_shape=True):\n",
    "    \"\"\"\n",
    "    将 nn.LSTM(bidir=True, num_layers=2) 的 ckpt 映射到我们的 LSTMCell BiLSTM。\n",
    "    直接使用 set_data 写入各个参数，避免名字前缀带来的 KeyError。\n",
    "    \"\"\"\n",
    "    st = ms.load_checkpoint(ckpt_path)\n",
    "    keys = set(st.keys())\n",
    "\n",
    "    def _as_np(x):  \n",
    "        return x.asnumpy() if hasattr(x, \"asnumpy\") else np.asarray(x)\n",
    "\n",
    "    emb_key = \"embedding.embedding_table\" if \"embedding.embedding_table\" in keys else \\\n",
    "              (\"embedding.weight\" if \"embedding.weight\" in keys else None)\n",
    "    if emb_key is not None:\n",
    "        np_val = _as_np(st[emb_key])\n",
    "        tgt = model.embedding.weight \n",
    "        if strict_shape:\n",
    "            assert tuple(tgt.shape) == tuple(np_val.shape), f\"embedding shape {tgt.shape} vs {np_val.shape}\"\n",
    "        tgt.set_data(ms.Tensor(np_val.astype(ms.dtype_to_nptype(tgt.dtype)), tgt.dtype))\n",
    "\n",
    "    for l in range(model.num_layers):\n",
    "        for is_rev, cells in [(False, model.fwd_cells), (True, model.bwd_cells)]:\n",
    "            suf = \"_reverse\" if is_rev else \"\"\n",
    "            cell = cells[l]\n",
    "            need = {\n",
    "                \"wih\": f\"rnn.weight_ih_l{l}{suf}\",\n",
    "                \"whh\": f\"rnn.weight_hh_l{l}{suf}\",\n",
    "                \"bih\": f\"rnn.bias_ih_l{l}{suf}\",\n",
    "                \"bhh\": f\"rnn.bias_hh_l{l}{suf}\",\n",
    "            }\n",
    "            for k in need.values():\n",
    "                if k not in keys:\n",
    "                    raise KeyError(f\"ckpt 缺少键: {k}\")\n",
    "\n",
    "            wih = _as_np(st[need[\"wih\"]]); whh = _as_np(st[need[\"whh\"]])\n",
    "            bih = _as_np(st[need[\"bih\"]]); bhh = _as_np(st[need[\"bhh\"]])\n",
    "\n",
    "            if strict_shape:\n",
    "                assert tuple(cell.weight_ih.shape) == tuple(wih.shape), f\"L{l}{suf} weight_ih {cell.weight_ih.shape} vs {wih.shape}\"\n",
    "                assert tuple(cell.weight_hh.shape) == tuple(whh.shape), f\"L{l}{suf} weight_hh {cell.weight_hh.shape} vs {whh.shape}\"\n",
    "                assert tuple(cell.bias_ih.shape)   == tuple(bih.shape), f\"L{l}{suf} bias_ih   {cell.bias_ih.shape} vs {bih.shape}\"\n",
    "                assert tuple(cell.bias_hh.shape)   == tuple(bhh.shape), f\"L{l}{suf} bias_hh   {cell.bias_hh.shape} vs {bhh.shape}\"\n",
    "\n",
    "            cell.weight_ih.set_data(ms.Tensor(wih.astype(ms.dtype_to_nptype(cell.weight_ih.dtype)), cell.weight_ih.dtype))\n",
    "            cell.weight_hh.set_data(ms.Tensor(whh.astype(ms.dtype_to_nptype(cell.weight_hh.dtype)), cell.weight_hh.dtype))\n",
    "            cell.bias_ih.set_data(ms.Tensor(bih.astype(ms.dtype_to_nptype(cell.bias_ih.dtype)),   cell.bias_ih.dtype))\n",
    "            cell.bias_hh.set_data(ms.Tensor(bhh.astype(ms.dtype_to_nptype(cell.bias_hh.dtype)),   cell.bias_hh.dtype))\n",
    "\n",
    "    for name in (\"fc.weight\", \"fc.bias\"):\n",
    "        if name in keys and hasattr(model.fc, name.split(\".\")[1]):\n",
    "            np_val = _as_np(st[name])\n",
    "            tgt = getattr(model.fc, name.split(\".\")[1])  # weight / bias\n",
    "            if strict_shape:\n",
    "                assert tuple(tgt.shape) == tuple(np_val.shape), f\"{name} {tgt.shape} vs {np_val.shape}\"\n",
    "            tgt.set_data(ms.Tensor(np_val.astype(ms.dtype_to_nptype(tgt.dtype)), tgt.dtype))\n",
    "\n",
    "    print(\"OK: 已把 nn.LSTM 的 ckpt 映射加载到 LSTMCell BiLSTM（直接 set_data，避免名字前缀问题）。\")\n",
    "load_lstm_ckpt_into_cell_bilstm(model, ckpt_path, strict_shape=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f5be370-edc3-4453-802a-f5eea34cd77f",
   "metadata": {},
   "source": [
    "## 自定义输入测试\n",
    "\n",
    "最后我们设计一个预测函数，实现开头描述的效果，输入一句评价，获得评价的情感分类。具体包含以下步骤:\n",
    "\n",
    "1. 将输入句子进行分词；\n",
    "2. 使用词表获取对应的index id序列；\n",
    "3. index id序列转为Tensor；\n",
    "4. 送入模型获得预测结果；\n",
    "5. 打印输出预测结果。\n",
    "\n",
    "具体实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b35902c6-84a3-4879-8fde-2d6f322f49eb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "score_map = {1: \"Positive\", 0: \"Negative\"}\n",
    "def predict_sentiment(net: nn.Cell, vocab: ds.text.Vocab, sentence: str):\n",
    "    tokens = sentence.lower().split()\n",
    "    ids = vocab.tokens_to_ids(tokens)\n",
    "    if isinstance(ids, int):\n",
    "        ids = [ids]\n",
    "    x = ms.Tensor(np.array(ids, dtype=np.int32)[None, :], ms.int32)\n",
    "    logits = net(x)                                   # (1,1) FP16\n",
    "    prob = mint.nn.functional.sigmoid(logits.astype(ms.float32)).asnumpy().item()\n",
    "    return score_map[int(prob >= 0.5)], prob"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68b87324-889b-4061-888c-4cd260f5c1d8",
   "metadata": {},
   "source": [
    "最后我们预测开头的样例，可以看到模型可以很好地将评价语句的情感进行分类。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4f6ea007-96a9-4b7c-9f5c-59329c0c7de9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('Negative', 0.06075190007686615)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict_sentiment(model, vocab, \"This film is terrible\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d6d9108e-8151-4ccc-a433-b33f437f1c24",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('Positive', 0.7727022171020508)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predict_sentiment(model, vocab, \"This film is great\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  },
  "vscode": {
   "interpreter": {
    "hash": "daf11ff21901010d76a413b98ce5bf702d1de4e9575dab37ba3694ffd67aa4ba"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
